2023-11-21 16:52:08.AIbase.3.4k
GPT-4's Accuracy on Graphic Reasoning Tasks is Only 33%, Sparking Concern
Research from the Santa Fe Institute shows that GPT-4 has only a 33% accuracy rate on graphic reasoning tasks. The multimodal version, GPT-4v, performs even worse with only a 25% accuracy rate, raising concerns about its graphic processing capabilities. Testing with the ConceptARC dataset reveals that the average accuracy of humans on graphic reasoning tasks reaches 91%, significantly higher than GPT-4. Researchers raised questions about the input methods used for participants and GPT-4, including sample randomness and how image transformations could alter concepts. The study highlights the limitations of large
